Image sensing apparatus

ABSTRACT

An Image sensing apparatus employing a multi-chip system and having a super-parallel circuit structure capable of performing processes such as image processing in real time. A first chip of a first of a first stage has first pixel circuits each having an optical sensor and first processing circuits and arranged in a matrix. A second chip of a second stage has second pixel circuits each having an analog memory for storing analog information from the preceding stage and second processing circuits and arranged in a matrix so as to correspond to the first pixel circuits. In each of the first and the second chip, each of the first and the second processing circuits receives an analog signal from another first and second processing circuit in the vicinity so as to perform first and second analog processing and performs circuit noise compensation by parallel calculation.

BACKGROUND OF THE INVENTION

The present invention relates to an image sensing apparatus, and moreparticularly to an image sensing apparatus called “vision chip”, whichis applied to a vision sensor and image processing. Here, the “visionchip” signifies, for example, a visual sensor chip (semiconductor orcircuit) in which a super-parallel circuit structure with optical sensorcircuits and processing circuits arranged for individual pixels isrealized by an analog CMOS integrated circuit, and which can performvisual information processing. Among such vision chips, especially avision chip which simulates the circuit structure and functions of anorganic retina shall be called “silicon retina”.

Shown in FIG. 22 is the schematic constructional view of a vision chip.A lower view is the enlarged view of the vision chip 61. As in theenlarged view, each pixel includes an optical sensor 63 and a processingcircuit 64. Image information is projected on the vision chip 61 throughan optical system constructed of a lens 62, etc. The projected imageinformation is converted into electric signals by the optical sensors63, and the electric signals are processed in super-parallel fashion bythe processing circuits 64 arranged in the respective pixels. Besides,the processing circuits 64 convert the electric signals into informationwhich is easily understood by a higher-degree recognition apparatus suchas computer or microprocessor, and which is outputted.

Owing to such a vision chip, a problem at which a prior-art imageprocessing system based on a serial type digital computer is inapt canbe solved. Various vision chips have heretofore been developed, and theprincipal items of visual information processing incarnated by thevision chips are, for example, “image smoothing”, “contour emphasis” and“motion detection”.

Shown in FIG. 23 is the circuit arrangement view of a single-chipsystem. Here, a vision chip which performs the smoothing, contouremphasis and velocity detection of an input image as one circuit examplewill be mentioned. In case of the single-chip system, besides an opticalsensor 63, the processing circuits of a smoothing circuit 65, a contouremphasis circuit 66 and a motion detection circuit 67 must berespectively arranged in each pixel as shown in the figure.

SUMMARY OF THE INVENTION

With the vision chip of the single-chip system as stated above, however,the optical sensor and the processing circuits are arranged in eachpixel, so that the geometrical structure (pixel size) of one pixelenlarges inevitably. Moreover, with the prior-art vision chip, variousvisual processing functions need to be incarnated by one chip, so thatthe number of pixels per unit area of the chip decreases to degrade aspatial resolution. Furthermore, with the prior-art vision chip, thepixel size is made small by simplifying the pixel structure, so thatsatisfactory image processing cannot be executed in higher-degreerecognition processing at the posterior stage in some cases. Besides,when a countermeasure in which the number of pixels is increased byenlarging a chip size is taken, the cost of the chip rises(incidentally, this is also ascribable to the fact that unusable chipsincrease with increase in the number of fabricated chips), and the chipsize itself has a limitation dependent upon a CMOS manufacturingprocess, from the viewpoint of available percentage.

In view of the above drawbacks, the present invention has for its objectto adopt the construction of a so-called “multi-chip system” in whichprocessing to be executed by one chip is divided and performed by aplurality of chips, whereby the various processing steps of imageprocessing or the like are performed by a super-parallel circuitstructure, and they are executed in real time. Also, the invention hasfor its object to provide a system of low price, small size and lowdissipation power utilizing, for example, CMOS. Further, the inventionhas for its object to provide a system having robustness peculiar toanalog devices.

According to the resolution means of the present invention, there isprovided an image sensing apparatus comprising:

-   -   first pixel circuits each of which includes an optical sensor        that converts an inputted light signal into an electric signal,        and a first processing circuit that executes first analog        processing for the output from the optical sensor and outputs        analog image information; and    -   second pixel circuits each of which includes a second analog        memory that receives the analog image information from the first        processing circuit of said first pixel circuit and stores the        received analog image information, and a second processing        circuit that reads out the image information from the second        analog memory, executes second analog processing and outputs        analog image information, and which is disposed in        correspondence with said first pixel circuit;    -   wherein the first and second pixel circuits being respectively        arranged in matrix shapes so as to form first and second chips,        and the first and second processing circuits respectively        receiving analog signals from the nearby first and second        processing circuits in the first and second chips corresponding        thereto, so as to compensate for characteristics and to execute        the first and second analog processing by parallel calculations.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a constructional view of a pixel sensing apparatus being amulti-chip system.

FIG. 2 is a block diagram of a type-1 chip.

FIG. 3 is a block diagram of a pixel circuit of Type 1.

FIG. 4 is a block diagram of a type-2 chip.

FIG. 5 is a block diagram of a pixel circuit of Type 2.

FIG. 6 is a timing chart of shift registers for selecting noticedpixels.

FIG. 7 is an arrangement diagram of the pixel circuit of one pixel ofType 1.

FIG. 8 is a timing chart concerning the operation of a pixel level.

FIG. 9 is an arrangement diagram of a resistance network.

FIG. 10 is an explanatory diagram of the image of contour emphasis basedon a unidimensional resistance network.

FIG. 11 is an explanatory diagram of the image of contour emphasis basedon resistance networks of two layers.

FIG. 12 is an arrangement diagram of the pixel circuit (2) of one pixelof Type 2.

FIG. 13 is a timing chart concerning the operation of a pixel level.

FIG. 14 is an explanatory diagram of the image of the difference betweenframes.

FIG. 15 is an arrangement diagram of the pixel circuit (1) of one pixelof Type 2.

FIG. 16 is a timing chart concerning the operation of a pixel level.

FIG. 17 is a constructional view of a multi-chip system.

FIG. 18 is a view of a binocular stereoscopic system constructed ofmulti-chip systems.

FIG. 19 is a circuit diagram of an active pixel sensor.

FIG. 20 shows a circuit diagram of a noise compensation buffer, and atiming chart of control signals.

FIG. 21 shows a circuit diagram and an explanatory diagram of aresistance network.

FIG. 22 is a schematic constructional view of a vision chip.

FIG. 23 is a circuit arrangement view of a single-chip system.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS OF THE INVENTION

1. Multi-Chip System

Shown in FIG. 1 is the constructional view of an image sensing apparatusbeing a multi-chip system. In this embodiment, by way of example, thethree processing items of smoothing, contour emphasis and motiondetection are respectively distributed to three chips, which areconstructed into a multi-chip. The smoothing chip 1 of first stage, towhich image information is inputted from an optical system 4 constructedof a lens, etc., includes an optical sensor circuit 1-1, and a smoothingcircuit 1-2 which is a processing circuit for smoothing, thereby tosmooth an input image. The smoothed image information is outputted asvoltage or current analog signals from the first-stage smoothing chip 1,and the analog signals are inputted to the contour emphasis chip 2 andmotion detection chip 3 of second stage, et seq. The pixels of thecontour emphasis chip 2 and motion detection chip 3 at the second stage,et seq. include analog memories 2-1, 3-1, and a contour emphasis circuit2-2 and a motion detection circuit 3-2 which are dedicated processingcircuits, respectively. The analog memories 2-1, 3-1 temporarily storethe analog information from the preceding stages, respectively. Thecontour emphasis circuit 2-2 and motion detection circuit 2-3 read theinformation of the analog memories 2-1, 3-1 (if necessary, they are alsocapable of writing information), respectively. Here, contour-emphasizedimage information items are outputted from the second-stage contouremphasis chip 2, while motion-detected results are outputted from thethird-stage motion detection chip 3.

As the features of such a multi-chip system, the following points, forexample, are mentioned:

-   -   Processing which is performed by one chip can be limited        (processing functions can be distributed).    -   A pixel size is small.    -   The number of pixels can be increased without enlarging a chip        size.    -   Since outputs are delivered in parallel from a plurality of        chips, high-degree image processing which utilizes a plurality        of visual information items can be executed.    -   A compensation circuit is easily arranged in each pixel in order        to remove circuit noise which is added in image processing and        data transfer.    -   Analog information is employed for image processing and the        transfer of data. Besides, the image processing can be executed        at high speed by a super-parallel circuit structure which is        based on analog processing circuits arranged in individual        pixels This point is clearly different from the operation of any        existing DSP (Digital Signal Processor) for image processing.

Here, the differences between the present invention and the DSP will bedescribed.

In case of the DSP, image information outputted from imaging equipmentsuch as a CCD camera is converted by an A/D converter into digitalsignals, which are sent to the DSP, and digital image processing isexecuted by the DSP. In contrast, in case of the embodiment of thepresent invention, at the stage of the first-stage smoothing chip 1,image information is initially acquired and is simultaneously subjectedto super-parallel analog image processing by the analog processingcircuits of the individual pixels (point of difference from the CCDcamera). Subsequently, computed analog information items are directlyoutputted from the first-stage smoothing chip 1 and are inputted to thecontour emphasis chip 2 of the next stage (A/D conversion is notperformed). Subsequently, the information items from the first-stagesmoothing chip 1 are inputted to the analog memories arrayed in therespective pixels of the next-stage contour emphasis chip 2 and arefurther subjected to super-parallel processing by the analog processingcircuits of the respective pixels (point of difference from digitalimage processing). Besides, information items from the contour emphasischip 2 are inputted to the analog memories arrayed in the respectivepixels of the next-stage motion detection chip 3 and are furthersubjected to super-parallel processing by the analog processing circuitsof the respective pixels.

In general, a processing circuit can be designed on a smaller scale inan analog circuit than in a digital circuit. Assuming that DSPs bearranged in individual pixels with the intention of performing parallelimage calculations, there are such problems as follows:

-   -   A pixel size becomes large.    -   Wiring for connecting respective pixels becomes complicated. In        contrast, according to the present invention, these problems are        relieved for the reason that the respective pixels are        constructed of analog circuits.

As a problem peculiar to an analog integrated circuit, however, theinfluence of dispersion ascribable to the non-uniformity of elementcharacteristics is supposed in some cases. This signifies that, evenwhen circuits being geometrically quite the same are designed within anidentical chip, the electrical characteristics of the individualcircuits become discrepant. In addition to the discrepancy appearingwithin the chip, the tendency appears more intensely among separatechips. The dispersion of the element characteristics among therespective pixels in the super-parallel circuit structure as in thepresent invention, and the change of the information items of thepreceding and succeeding chips in the case of the transfer of imageinformation to the next stage need to take countermeasures when they areserious.

In the present invention, therefore, the problem is solved in such a waythat a circuit which compensates for the dispersion (circuit noise)attendant upon the non-uniformity of the element characteristics isincorporated in each pixel. In the compensation circuit, the circuitnoise of the corresponding pixel is stored in a certain sort of analoginformation storage element, thereby to incarnate image processing anddata transfer which are not influenced by the circuit noise. Concretely,a noise compensation buffer circuit (to be explained later) is supposed,but any circuit having a similar function can be substituted therefor.

2. Outline of Circuit Arrangement of Chips Constituting Multi-chipsystem

The sorts of chips constituting a multi-chip system are broadly dividedinto the following two types in the embodiment of the present invention:

-   -   Type 1: an optical sensor for acquiring image information, and a        processing circuit are built in each pixel    -   Type 2: an analog memory for storing image information from a        preceding stage, and a processing circuit are built in each        pixel

Circuit arrangements will be described on the types 1 and 2.

(1) Circuit Arrangement of Type-1 Chip (Image Acquisition+Parallel ImageProcessing)

The chip of Type 1 is at the first stage of the chip systemto-be-constructed, and it acquires the image information items of theoutside world by the optical sensors built in the respective pixels andsubjects the acquired information items to image processing in parallel.

Shown in FIG. 2 is the block diagram of the type-1 chip.

The type-1 chip includes pixel circuits 11, a horizontal shift register13, a vertical shift register 14, switches 15, an outputting buffer 16,and an output line 17. The pixel circuits 11 are sequentially selectedby the horizontal shift register 13 and the vertical shift register 14,so as to read out outputs calculated in parallel. (Incidentally, thetiming chart of the shift registers will be explained later.)

Shown in FIG. 3 is the block diagram of the pixel circuit of Type 1. Thepixel circuit 11 includes an optical sensor 111, a processing portion112, an analog calculator 113, and a switch 114. The roles of therespective constituents will be described below.

The optical sensor 111 converts an optical signal (image information)into an electric signal such as voltage or current. Used as a circuitelement therefor is any of a photodiode, a phototransistor, an activepixel sensor, etc. The processing portion 112 receives an input from theoptical sensor 111 of its own pixel circuit, and inputs n1-n4 from thenearby pixel circuits, and it performs image processing. This processingportion 112 chiefly performs parallel image calculations which employthe information items of the nearby pixels. The analog calculationportion 113 receives inputs from the processing portion 112, andperforms an analog calculation conforming to the four rules ofarithmetic, or the like. This analog calculation portion 113 may wellsimultaneously include a circuit which compensates for circuit noiseascribable to the dispersion of the characteristics of the individualelements, the circuit noise being a problem that is to be dealt with inan analog integrated circuit. The output of the analog calculationportion 113 becomes a pixel output. In this embodiment, a control signalfor the pixel circuits 11 is inputted to all the pixels collectively,but it may well be inputted every pixel circuit or every row or column.

(2) Circuit Arrangement of Type-2 Chip (Analog Memory+Parallel ImageProcessing)

The chip of Type 2 is at the second or subsequent stage of the chipsystem to-be-constructed, and it stores image information itemsdelivered from the preceding stage, in the analog memories built in therespective pixels, and subjects the delivered information items to imageprocessing in parallel by the processing circuits arranged in therespective pixels.

Shown in FIG. 4 is the block diagram of the type-2 chip.

The type-2 chip includes pixel circuits 21, a horizontal shift register23, a vertical shift register 24, switches 25, an inputting buffer 26,an output line 27, an input line 28, and an outputting buffer 29.Noticed pixels are sequentially selected by the horizontal shiftregister 23 and the vertical shift register 24, so as to input data fromthe chip of the preceding stage to the pixel circuits 21 and to read outoutputs calculated in parallel. (Incidentally, the timing chart of theshift registers will be explained later.)

Shown in FIG. 5 is the block arrangement of the pixel circuit of Type 2.

The pixel circuit 21 is constructed of an analog memory 211, aprocessing portion 212, an analog calculator 213, and it includesswitches 114, 115. The roles of the respective constituents will bedescribed below.

The analog memory 211 stores image information inputted from outside (inthis case, the pixel circuit of Type 1, the pixel circuit of Type 2 atthe preceding stage, or the like). The processing portion 212 receivesan input from the analog memory 211 of its own pixel circuit, and inputsn1-n4 from the nearby pixel circuits, and it performs image processing.This processing portion 212 chiefly performs parallel image calculationswhich employ the information items of the nearby pixels. The analogcalculation portion 213 receives inputs from the processing portion 212,and performs an analog calculation conforming to the four rules ofarithmetic, or the like. This analog calculation portion 213 may wellsimultaneously include a circuit which compensates for circuit noiseascribable to the dispersion of the characteristics of the individualelements, the circuit noise being a problem that is to be dealt with inan analog integrated circuit. The output of the analog calculationportion 213 becomes a pixel output. In this embodiment, a control signalfor the pixel circuits is inputted to all the pixels collectively, butit may well be inputted every pixel circuit or every row or column.

Next, shown in FIG. 6 is the timing chart of the shift registers forselecting noticed pixels.

In both the chips of Types 1 and 2, the horizontal shift registers 13,23 and the vertical shift registers 14, 24 select noticed pixel circuitsto/from which image information items are inputted/outputted. The basictimings of the shift registers become as shown in the figure. In thechip of Type 2, the row of a pixel array is selected by the verticalshift register 24, and the input/output switch SWi/o of the selectedpixels is connected to the input/output lines of the respective columns.In this state, the input/output lines of one set are selected by thehorizontal shift register 23 and the switch 25, and they arerespectively connected to the input buffer 26 and the output buffer 29.That is, the input buffer 26 and the output buffer 29 are connected tothe pixel selected by the row and the column. A situation in the case ofType 1 is similar, but the path of an input is unnecessary.

3. Circuit Examples of Chips

(1) Circuit Example of Type-1 Chip: Image Acquisition+Smoothing

Shown in FIG. 7 is the arrangement diagram of the pixel circuit of onepixel of Type 1.

This pixel circuit includes an optical sensor 111, a processing portion112, an analog calculator 113, and a switch 114. In this example, theoptical sensor 111 is made an APS (active pixel sensor: to be explainedlater), which accumulates light charges and thereby converts a lightsignal into voltage information. The processing portion 112 isconstructed of a resistance network here. Image information is inputtedto the resistance network is formed by having resistance-basedconnections (n1, n2, n3, n4) with nearby pixels, so as to smooth theinput image. An output from the processing portion 112 is inputted tothe analog calculator 113. The analog calculator 113 employs a noisecompensation buffer circuit Nbuf: to be explained later. The noisecompensation buffer circuit Nbuf can compensate for the dispersion ofthe circuits on the input side and the offset of an amplifier withinthis noise compensation buffer circuit Nbuf, in accordance with controlsignals. A switch SWo is controlled by the horizontal shift register 13and the vertical shift register 14, whereby a noticed pixel can beselected to read out data.

Shown in FIG. 8 is a timing chart concerning the operation of a pixellevel. Now, operations will be described as to individual intervals.

Interval (A): A control signal SWp for the switch of the APS is assertedto “H”, thereby to initialize the APS. Thereafter, the control signalSWp is negated to “L”, thereby to shift to a charge accumulatingoperation in the APS for the nth frame.

Intervals (B)+(C): Accumulation time of the APS

Interval (C): With the lapse of the accumulation time, a control signalSWh is asserted to “H”, thereby to connect the resistance network of theprocessing portion 112 and the noise compensation buffer circuit Nbuf.On this occasion, the output Vnet of the resistance network becomes:Vnet=Vnet(n)+VN 1  (1)Here, Vnet(n) denotes image information processed by the resistancenetwork, and VN1 denotes circuit noise at this time. Control signals SW1and SW2 are controlled in this state, whereby the output of theresistance network as expressed by Eq. (1) is stored in a capacitancebuilt in the noise compensation buffer circuit Nbuf.

Interval (A′): The control signal SWP for the switch of the APS isasserted to “H” again, thereby to initialize the APS. On this occasion,the output Vnet of the resistance network becomes:Vnet=Vnet 0+VN 0  (2)Here, Vnet0 denotes an initial voltage from the resistance network atthe initialization of the APS, and VN0 denotes circuit noise at theinitialization. That is, an input voltage to the noise compensationbuffer circuit Nbuf has changed from Eq. (1) to Eq. (2). On thisoccasion, assuming that the magnitude of the circuit noise be alwaysconstant (VN1=VN0), an output Vout from the noise compensation buffercircuit Nbuf becomes: $\begin{matrix}{{{Vout}\quad(n)} = {{{Vnet}\quad(n)} - {Vnet0} + {VN1} - {VN0} + {Vref}}} \\{= {{{Vnet}\quad(n)} - {Vnet0} + {Vref}}}\end{matrix}$so that the output which is proportional to processed information in theresistance network as is free from the influence of the circuit noise isobtained. Thereafter, the control signal SWh is asserted to “H”, wherebythe output is held in the noise compensation buffer circuit Nbuf.Besides, the control signal SWp is negated to “L”, thereby to shift to acharge accumulating operation in the APS for the (n+1)th frame.

Interval (B′): Since the APS and the circuit Nbuf are electricallyisolated, the output Vout(n) can be read out by closing an output switchsignal SWo, in parallel with the accumulating operation in the APS forthe (n+1)th frame.

The acquisition of image information and the smoothing operations basedon the resistance networks can be performed by repeating the aboveoperations.

(2) Circuit Example 1 of Type-2 Chip: Contour Emphasis

Shown in FIG. 9 is the arrangement diagram of a resistance network.

A network in which pixels are coupled by resistances as in the figure,is called the “resistance network”. Owing to the resistance network, aninput image can be smoothed (as will be explained later). Thedifferential output between the input image and a smoothed imageemphasizes the contour of the input image.

Shown in FIG. 10 is the explanatory diagram of the image of contouremphasis based on a unidimensional resistance network.

The axis of abscissas in Diagram (A) represents a pixel No., while theaxis of ordinates represents corresponding pixel information (voltage).An input Vk whose voltage value changes greatly at the 0th pixel, isapplied to the resistance network. This corresponds to the contour of animage. On this occasion, a signal V1 k with a contour part smoothed isoutputted from the resistance network. It is in Diagram (B) that thedifferential output between the signals Vk and V1 k has been taken. Itis seen that a great response is exhibited at the 0th pixel being theposition of the contour, and that, around the 0th pixel, the voltagegradually comes to take a constant value with a distance from theposition of the contour. That is, the contour part is emphasized.

Besides, shown in FIG. 11 is the explanatory diagram of the image ofcontour emphasis based on resistance networks of two layers.

When image information smoothed by another resistance network beforehandas stated above is employed as the input of a resistance network, thatis, when resistance networks are used in two layers, an output as in thefigure is obtained. It is known that the characteristics of the filterapproximate Laplacian-Gaussian (∇²G) mathematically, and the smoothingof an input image and the emphasis of a contour can be simultaneouslyperformed. Moreover, the filter is excellent for removing spatial imagenoise at high frequencies.

Shown in FIG. 12 is the arrangement diagram of the pixel circuit (2) ofone pixel of Type 2.

This pixel circuit includes an analog memory 211, a processing portion212, an analog calculator 213, and switches 214, 215. The analog memory211 stores external image information in a capacitor built therein. Theprocessing portion 212 is constructed of a resistance network. Thisprocessing portion 212 forms the resistance network by havingresistance-based connections (n1, n2, n3, n4) with nearby pixels, so asto smooth an input image. Both an input to and an output from theprocessing portion 212 are outputted to the analog calculator 213. Anoise compensation buffer circuit Nbuf (to be explained later) isemployed as the analog calculator 213. The noise compensation buffercircuit Nbuf can compensate for the dispersion of the circuits on theinput side and the offset of an amplifier within this noise compensationbuffer circuit Nbuf, in accordance with control signals. Signals SWi,SWo are controlled by the horizontal shift register 23 and the verticalshift register 24, whereby a noticed pixel can be selected.

Shown in FIG. 13 is a timing chart concerning the operation of a pixellevel. Although one layer of the resistance network will be exemplifiedhere, two layers may well be employed as stated above. Now, operationswill be described as to individual intervals.

Interval (A): An input control signal SWi from the shift register isasserted to “H”, thereby to store external image information (Vin(n)) inthe analog memory 211.

Interval (B): The control signal SWh of a data holding switch and thecontrol signal SWs of an input changeover switch are asserted to “H”,thereby to connect the noise compensation buffer circuit Nbuf and aninput V1 from the analog memory 211. The switching operations of thecontrol signals SW1, SW2 of the noise compensation buffer circuit Nbufare performed in this state, whereby the input V1 from the analog memory211 is stored in the noise compensation buffer circuit Nbuf.Simultaneously, the dispersion of the input side of the circuit and theoffset of the amplifier within the noise compensation buffer circuitNbuf are compensated for (noise compensation operation).

Interval (C): The control signal SWs of the changeover switch is negatedto “L”, thereby to connect the noise compensation buffer circuit Nbufand the output V2 of the resistance network, so that the followingcomputation is executed in the noise compensation buffer circuit Nbuf:Vout(n)=V 2(n)−V 1(n)+VrefThat is, an output which is proportional to the difference between theinput image information V1(n) and the smoothed output V2(n) from theresistance network is obtained.

Interval (D): The control signal SWh of the data holding switch isnegated to “L”, whereby the computed image information is held in thenoise compensation buffer circuit Nbuf.

Interval (A′): An output control signal SWo from the shift register isasserted to “H”, thereby to readout the output Vout(n) held in the noisecompensation buffer circuit Nbuf. Simultaneously therewith, the inputcontrol signal SWi from the shift register is asserted to “H”, wherebyimage information (Vin(n+1)) at the next time is stored in the analogmemory 211.

Thenceforth, similar operations are repeated, whereby acontour-emphasized output can be delivered.

(3) Circuit Example 2 of Type-2 Chip: Motion Detection

Shown in FIG. 14 is the explanatory diagram of the image of thedifference between frames.

It is assumed that, in the image of the certain nth frame, a black dischave been entered in a white ground (Diagram (A)). It is supposed thatthe black disc have moved rightwards in the image of the (n+1)th frame(Diagram (B)). It is assumed that the image outputs of the nth and(n+1)th frames be voltage values, and the potential difference betweenthese outputs is computed. Then, only the part of the movement respondsas in Diagram (C). In this manner, the motion of an object can becomputed by computing the difference between the frames.

Shown in FIG. 15 is the arrangement diagram of the pixel circuit (1) ofone pixel of Type 2.

This pixel circuit includes an analog memory 211, an analog calculator213, and switches 214, 215. The analog memory 211 stores external imageinformation in a capacitor built therein. A noise compensation buffercircuit Nbuf (: to be explained later) is employed as the analogcalculator 213. The noise compensation buffer circuit Nbuf cancompensate for the dispersion of the circuits on the input side and theoffset of an amplifier within this noise compensation buffer circuitNbuf, in accordance with control signals. Signals SWi, SWo arecontrolled by the horizontal shift register 23 and the vertical shiftregister 24, whereby a noticed pixel can be selected.

Shown in FIG. 16 is a timing chart concerning the operation of a pixellevel. Now, operations will be described as to individual intervals.

Initial condition: A signal SWh is always held at an “H” input, therebyto connect the analog memory 211 and the noise compensation buffercircuit Nbuf.

Interval (A): A pixel value (Vin(n)) on the analog memory 211 is storedin the noise compensation buffer circuit Nbuf by the switchingoperations of the control signals SW1, SW2 of this noise compensationbuffer circuit Nbuf. Simultaneously, the dispersion of the input side ofthe circuit and the offset of the amplifier within the noisecompensation buffer circuit Nbuf are compensated for (noise compensationoperation).

Interval (B): An input control signal SWi from the shift register isasserted to “H”, whereby the information of the analog memory 211 isupdated to information at the next time (Vin(n+1)). Simultaneously, thefollowing computation is executed in the noise compensation buffercircuit Nbuf:Vout=Vin(n)−Vin(n+1)+VrefThat is, an output which is proportional to the difference between thepixel information at the current time (n+1) and the pixel information atthe last time (n) is obtained. An output control signal SWo is assertedto “H”, thereby to read out the output of the noise compensation buffercircuit Nbuf.

Interval (A′): The switching operations of the control signals SW1, SW2are performed again, whereby pixel information (Vin(n+1)) on the analogmemory 211 is stored in the noise compensation buffer circuit Nbuf, andthe noise compensation operation is performed by the noise compensationbuffer circuit Nbuf.

Thenceforth, similar operations are repeated, whereby the differentialoutput between frames can be delivered.

4. Constructional Examples of Multi-Chip Systems

Shown in FIG. 17 is an example of the constructional view of amulti-chip system.

Here, the following chips mentioned as the chip circuit examples aboveare constructed in the form of the multi-chip system:

-   -   Image acquisition+smoothing chip 1    -   Contour emphasis chip 2    -   Motion detection chip 3

Here will be described a circuit example in which the imageacquisition+smoothing chip 1, the contour emphasis chip 2 and the motiondetection chip 3 are connected in series in this order.

First, the acquisition of an input image and the smoothing of the inputimage are performed in the smoothing chip 1 of the initial stage. Whenthe image of an apple has been projected on the smoothing chip 1 asshown in Diagram (A), an output in which spatial noise componentscontained in the image are smoothed is obtained.

The output of the initial-stage smoothing chip 1 is inputted to thecontour emphasis chip 2 of the second stage. Since contour emphasisprocessing with the smoothed image is executed in the second-stagecontour emphasis chip 2, a filter of Laplacian-Gaussian type is formed,and an output in which the smoothing and contour emphasis of the inputimage are done is obtained. It is understood that, as shown in Diagram(B), feature quantities such as the contour of the apple and the stringsof leaves are emphasized.

The output of the second-stage contour emphasis chip 2 is inputted tothe motion detection chip 3 of the third stage. A part where the imagewith the contour of the input image emphasized has moved, can bedetected in the third-stage motion detection chip 3. When the apple hashorizontally moved rightwards, it is understood that, as in Diagram (C),the output of the contour in the direction in which the apple movesappears low (in black), whereas the output of the contour on theopposite side appears high (in white), almost no response beingexhibited in the vertical direction in which the apple does not move.

Owing to such construction of the chip system, the “smoothed image”,“contour-emphasized image” and “motion image” can be outputted inparallel.

Next, shown in FIG. 18 is an example of the constructional view of abinocular stereoscopic system constructed of multi-chip systems.

Even a complicated image processing system such as of binocularstereoscopy can be coped with by preparing two chips 51, 52 of Type 1for acquiring images as shown in the figure.

The outputs of the two type-1 chips 51, 52 are inputted to a pluralityof visual function chips 53, 54, 55, 56 so as to derive visualinformation items in parallel. Thereafter, the information items arebound. This permits to precisely and quickly solve the correspondingpoint problem at which a general serial image processing system isinapt.

Here, the visual function chips 53, 56 function as the motion detectionchips stated above, and the visual function chips 54, 55 function as thecontour emphasis chips stated above. Besides, a visual function chip 57applies the motion detection chip and receives the outputs of the twovisual function chips 54, 55, thereby to have the function of detectingthe difference of the outputs.

5. Circuit Examples

Shown in FIG. 19 is an example of the circuit diagram of an active pixelsensor.

In this example, a photosensor is used in a charge accumulating mode,and a source-follower circuit is added to the output node of thephotosensor, thereby to construct the active pixel sensor (APS). Sincethe output of the photosensor lowers to the amount of the thresholdvoltage of MOS at initialization, a PMOS source follower (PSF) isemployed as the source-follower circuit.

Shown in FIG. 20 are examples of the circuit diagram of a noisecompensation buffer and the timing chart of control signals.

A noise compensation buffer circuit is a circuit in which circuit noiseascribable to the dispersion of elements is stored in a built-incapacitance, thereby to compensate for the circuit noise (refer to T.Sibano, K. Iizuka, M. Miyamoto, M. Osaka, R. Miyama and A. Kito:“Matched Filter for DS-CDMA of up to 50 MChip/s Based on Sampled AnalogSignal Processing”, ISSCC Digest of Tech. Papers, pp. 100-101, February1997).

Operations will be described in conjunction with the timing chart ofcontrol signals in the figure.

[1] (Sw1: ON, Sw2: Connected to Ref)

On this occasion, input V(in)=Vin0+VN0 (here, Vin0: initial value of aninput signal from a circuit at a preceding stage, VN0: circuit noise ofthe preceding stage)

Voltage of the inverting node of an AMP,V(in−)=V(ref)+Voff(here, Voff: offset voltage of the AMP)

Charges accumulated in the inverting node of the AMP,Q=C 1(V(ref)+Voff−Vin 0−VN 0)+C 2(V(ref)+Voff−V(ref))  (3)[2] (Sw1: Off, Sw2: Connected to Ref)

On this occasion, the inverting node of the AMP falls into a floatingstate, and the charges are held as they are. (An operation till here iscalled the “reset operation”.) [3] (SW1: OFF, SW2: connected to theoutput of the AMP)

On this occasion, the input changes to V(in)=Vin1+VN1 (here, Vin1: inputsignal from the circuit at the preceding stage, VN1: circuit noise ofthe preceding stage at the input of the signal Vin1)

Charges accumulated in the inverting node of the AMP,Q=C 1(V(ref)+Voff−Vin 1−VN 1)+C 2(V(ref)+Voff−V(out))  (4)From Eqs. (3) and (4),V(out)=−(C 1/C 2)(Vin 1−Vin 0+VN 1−VN 0)+V(ref)

Accordingly, if the circuit noise in the preceding-stage circuit isconstant (VN0=VN1), the output V(out) is free from the influence of, notonly the offset of the AMP of the compensation buffer circuit itself,but also the circuit noise of the preceding stage, and it becomesproportional to the variation of the input voltage.

Incidentally, the operating region of the noise compensation buffercircuit depends upon the operating region of the AMP employed. Theoperating regions for the respective AMPs used are as follows:

-   -   Transconductance amplifier (AMP1): Operating for the minus        change of the input    -   Transconductance amplifier (AMP2): Operating for the plus change        of the input    -   Wide-range amplifier: Operating for both the plus and minus        changes of the input

Shown in FIG. 21 are the circuit diagram and explanatory diagram of aresistance network.

Super-parallel image calculations based on the resistance network willbe described with reference to this figure (refer to C. Mead: “AnalogVLSI and Neural Systems”, Addision-Wesley, Reading, Mass., 1989, and T.Yagi, S. Ohshima and Y. Funahashi: “The role of retinal bipolar cell inearly vision: an implication with analogue networks and regularizationtheory”, Biol. Cybern, 77, pp. 163-171, 1997). The distribution of theoutput voltages of the resistance network becomes the input voltagesthereof as smoothed. In a case where the number of nodes is sufficientlylarge, the voltage distribution in the resistance network can beconsidered as stated below.

The voltage distribution will be explained in conjunction with theschematic diagram of the resistance network in Diagram (A).

It is assumed that the input voltage at k=0 be Vk=V0, and that the otherinput voltages be zero (spatial impulse input). The response potentialV1 k of the resistance network to this input becomes:V 1 _(k) =B ₁ V _(0γ1) ^(|k|)  Equation 1

-   -   Here, B₁₌1/{square root}{square root over (4L₁ ²+1)} L₁={square        root}{square root over (R_(m)/R_(s))}        γ₁=1+1/(2 L ₁ ²)−{square root}{square root over (1/L ₁ ²+1/(4 L        ₁ ⁴))}        This equation represents that the output signal attenuates        exponentially with a distance from the signal source (k=0).

Besides, L1 is called the “spatial constant of the resistance network”,and the signal propagates more extensively as the numerical value of thespatial constant L1 is larger.

Shown in Diagram (B) are results obtained in such a way that the outputvoltage distribution at the spatial impulse input was computed for twosorts of spatial constants. A solid line corresponds to L1={squareroot}(10/6), while a broken line corresponds to {square root}(10/1).Both the results smoothly attenuate exponentially with the distance fromthe signal source. It is also understood that the output propagates moreextensively in the broken line of the larger spatial constant.

A response to any desired input potential distribution Vi is expressedby the spatial convolution integral of a response in the case of puttingVk=1 in Equation 1 and the distribution Vi, as follows:$V_{1k} = {B_{1}\quad{\sum\limits_{i = {- \infty}}^{\infty}{V_{i}\quad\gamma_{1}^{{k - i}}}}}$

That is, in case of utilizing the resistance network as a constituentcircuit for image processing,

-   -   it can execute the smoothing processing of an input image at        high speed in super-parallel fashion by arraying optical sensors        or pixel memories in an input portion, and    -   it can freely regulate a region to-be-smoothed by constructing        resistance elements out of variable resistances.

INDUSTRIAL APPLICABILITY

As described above, the present invention adopts the construction of aso-called “multi-chip system” in which processing intended to beexecuted by one chip is divided and performed by a plurality of chips,whereby the various processing steps of image processing or the like areperformed by a super-parallel circuit structure, and they can beexecuted in real time. Moreover, according to the present invention, asystem of low price, small size and low dissipation power, whichutilizes CMOS by way of example can be provided. Furthermore, accordingto the present invention, a system which has robustness peculiar toanalog devices can be provided.

1. An image sensing apparatus, comprising: first pixel circuits each ofwhich includes an optical sensor that converts an inputted light signalinto an electric signal, and a first processing circuit that executesfirst analog processing for the output from the optical sensor andoutputs analog image information; and second pixel circuits each ofwhich includes a second analog memory that receives the analog imageinformation from the first processing circuit of said first pixelcircuit and stores the received analog image information, and a secondprocessing circuit that reads out the image information from the secondanalog memory, executes second analog processing and outputs analogimage information, and which is disposed in correspondence with saidfirst pixel circuit; wherein the first and second pixel circuits beingrespectively arranged in matrix shapes so as to form first and secondchips, and the first and second processing circuits respectivelyreceiving analog signals from the nearby first and second processingcircuits in the first and second chips corresponding thereto, so as tocompensate for characteristics and to execute the first and secondanalog processing by parallel calculations.
 2. An image sensingapparatus as defined in claim 1, further comprising third pixel circuitseach of which includes a third analog memory that receives the analogimage information from the second processing circuit of said secondpixel circuit and stores the received analog image information, and athird processing circuit that reads out the image information from thethird analog memory, executes third analog processing and outputs analogimage information, and which is disposed in correspondence with saidfirst and second pixel circuits; wherein the third pixel circuits beingarranged in a matrix shape so as to form a third chip, and the thirdprocessing circuits respectively receiving analog signals from thenearby third processing circuits in the third chip so as to compensatefor characteristics and to execute the third analog processing byparallel calculations.
 3. An image sensing apparatus as defined in claim2, characterized in: that the first chip executes image acquisition andsmoothing processing for acquired image information; that the secondchip executes contour emphasis processing for image information fromsaid first chip; and that the third chip executes motion detectionprocessing for image information from said second chip.
 4. An imagesensing apparatus as defined in claim 1, wherein the first chip furthercomprises: a horizontal shift register and a vertical shift registerwhich sequentially select the first pixel circuits and read out outputscalculated in parallel; switches each of which selects the analog outputfrom any of said first pixel circuits; and an outputting buffer for theanalog output selected by said each switch.
 5. An image sensingapparatus as defined in claim 1, wherein the second chip furthercomprises: a horizontal shift register and a vertical shift registerwhich sequentially select the second pixel circuits and read out outputscalculated in parallel; switches each of which selects the analog outputfrom any of said second pixel circuits; an outputting buffer whichtemporarily accumulates the analog output selected by said each switch;and an inputting buffer for the analog input to said each second pixelcircuit.
 6. An image sensing apparatus as defined in claim 1, wherein insaid first pixel circuit: the optical sensor includes an active pixelsensor which converts the light signal into the electric signal inaccordance with a control signal; and the first processing circuitincludes: a resistance network which receives an analog signal from theactive pixel sensor of said first pixel circuit of its own, and analogsignals from the nearby first pixel circuits, so as to smooth an analoginput image from said active pixel sensor; a noise compensation buffercircuit which receives an analog signal from the resistance network andperforms an analog calculation, so as to compensate for a dispersion ofcircuit element characteristics on an input side and an offset of aninternal amplifier in accordance with control signals, thereby tocompensate for circuit noise; and a switch for outputting an analogsignal from the noise compensation buffer circuit; wherein said firstpixel circuit performs acquisition of an image signal and smoothingprocessing.
 7. An image sensing apparatus as defined in claim 1,wherein: said second pixel circuit further includes a first switch whichcontrols input of a pixel signal from said first pixel circuit; thesecond analog memory stores an analog image in an internal capacitor;and the second processing circuit includes: a resistance network whichreceives an analog signal from the second analog memory of said secondpixel circuit of its own, and analog signals from the nearby secondpixel circuits, so as to smooth an analog input image from said secondanalog memory; a noise compensation buffer circuit which receives analogsignals of the input and output of the resistance network aschanged-over in accordance with control signals and performs an analogcalculation, so as to compensate for a dispersion of circuit elementcharacteristics on an input side and an offset of an internal amplifier,thereby to compensate for circuit noise; and a switch for outputting ananalog signal from the noise compensation buffer circuit; wherein saidsecond pixel circuit performs contour emphasis processing.
 8. An imagesensing apparatus as defined in claim 2, wherein: said third pixelcircuit further includes a first switch which controls the reception ofa pixel signal from said second pixel circuit; the third analog memorystores an analog image in an internal capacitor; and the thirdprocessing circuit includes: a noise compensation buffer circuit whichreads out image information from the analog memory and outputs an analogsignal proportional to a difference between the current imageinformation and the last image information, in accordance with controlsignals, and which compensates for a dispersion of circuit elementcharacteristics on an input side and an offset of an internal amplifier,thereby to compensate for circuit noise; and a second switch foroutputting the analog signal from the noise compensation buffer circuit;wherein said third pixel circuit performs motion detection processing.9. An image sensing apparatus as defined in claim 2, comprising: a chipfor a left eye and a chip for a right eye, each of which includes thefirst chip that executes image acquisition and smoothing processing, thesecond chip that executes motion detection processing for an output ofsaid first chip, and the third chip that executes contour emphasisprocessing, and which execute image processing corresponding to the lefteye and the right eye, respectively; a fourth chip which executesprocessing for computing parallax, for outputs from the second chipscorresponding to the left eye and the right eye; and a binding portionwhich binds outputs of the first to fourth chips.
 10. An image sensingapparatus as defined in claim 2, wherein the first chip furthercomprises: a horizontal shift register and a vertical shift registerwhich sequentially select the first pixel circuits and read out outputscalculated in parallel; switches each of which selects the analog outputfrom any of said first pixel circuits; and an outputting buffer for theanalog output selected by said each switch.
 11. An image sensingapparatus as defined in claim 2, wherein the second chip furthercomprises: a horizontal shift register and a vertical shift registerwhich sequentially select the second pixel circuits and read out outputscalculated in parallel; switches each of which selects the analog outputfrom any of said second pixel circuits; an outputting buffer whichtemporarily accumulates the analog output selected by said each switch;and an inputting buffer for the analog input to said each second pixelcircuit.
 12. An image sensing apparatus as defined in claim 2, whereinin said first pixel circuit: the optical sensor includes an active pixelsensor which converts the light signal into the electric signal inaccordance with a control signal; and the first processing circuitincludes: a resistance network which receives an analog signal from theactive pixel sensor of said first pixel circuit of its own, and analogsignals from the nearby first pixel circuits, so as to smooth an analoginput image from said active pixel sensor; a noise compensation buffercircuit which receives an analog signal from the resistance network andperforms an analog calculation, so as to compensate for a dispersion ofcircuit element characteristics on an input side and an offset of aninternal amplifier in accordance with control signals, thereby tocompensate for circuit noise; and a switch for outputting an analogsignal from the noise compensation buffer circuit; wherein said firstpixel circuit performs acquisition of an image signal and smoothingprocessing.
 13. An image sensing apparatus as defined in claim 2,wherein: said second pixel circuit further includes a first switch whichcontrols input of a pixel signal from said first pixel circuit; thesecond analog memory stores an analog image in an internal capacitor;and the second processing circuit includes: a resistance network whichreceives an analog signal from the second analog memory of said secondpixel circuit of its own, and analog signals from the nearby secondpixel circuits, so as to smooth an analog input image from said secondanalog memory; a noise compensation buffer circuit which receives analogsignals of the input and output of the resistance network aschanged-over in accordance with control signals and performs an analogcalculation, so as to compensate for a dispersion of circuit elementcharacteristics on an input side and an offset of an internal amplifier,thereby to compensate for circuit noise; and a switch for outputting ananalog signal from the noise compensation buffer circuit;